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1. Introduction 

Quotients modulo nonreductive groups arise in the study of moduli spaces of 
sheaves. Let k be an algebraically closed field of characteristic zero. We fix coherent 
algebraic sheaves £ and T on the projective space P r = W(k), which are direct 
sums of simple sheaves. The linear algebraic group G = Aut(£) x Aut(F) acts 
by conjugation on the finite dimensional vector space W = Hom(£, J 7 ). To every 
character of G Drezet and Trautmann associate subsets W ss C W of semistable 
points and W s C W of stable points. One expects there to be categorical quotients 
W ss //G and W s /G. In [4] the author has found many examples of locally closed 
subsets inside moduli spaces of semistable sheaves (in the sense of Gieseker) on 
P 2 , with linear Hilbert polynomial, that are isomorphic to quotients of the form 
W ss //G. 

This paper is not concerned with the study of moduli spaces but only with the 
question of existence of quotients modulo G. If £ has more than one kind of simple 
sheaf in its decomposition, then Aut(£) is nonreductive, so this situation falls out- 
side Mumford's Geometric Invariant Theory. Drezet and Trautmann address this 
difficulty in [3] by embedding the action of G on W into the action of a reductive 
group G onto a larger space W. Their main result states that if certain compat- 
ibility conditions relating the semistable points of W and W are satisfied, then 
W ss I /G and W s /G exist and are projective, respectively quasiprojective varieties. 
Moreover, they find sufficient conditions, expressed in terms of linear algebra con- 
stants, under which the compatibility conditions are fulfilled. This allows Drezet 
and Trautmann to establish the existence of quotients for certain classes of mor- 
phisms, notably for morphisms of the form 

m x O(-2) © m 2 0(-l) — > nO, 
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cf. 6.4 in [2]. The purpose of this paper is to give more examples to the Drezet- 
Trautmann theory. We use their embedding into the action of G but we do not use 
their linear algebra constants. We are concerned only with the geometric quotients 
W s /G and we do not discuss properly semistable morphisms, i.e. morphisms which 
are semistable but not stable. Applying 6.6.1 from [3] we establish the existence of 
geometric quotients in the following situations: 

• mO(-di) © 20(-d 2 ) — ► nO on P r such that < Ai < l/(2o + m) and 
either the conditions 

n.<( r - 1+ *-*V ^-^(.-f*- 1 

\ r — 1 / a + m — 1 \ n \ r 

or the conditions 

/r + di - d 2 \ . ^ 1 2 /r + d 2 - 1 
to < , Ai < 7 



r / 3to 3mn \ r 

are satisfied. Here m and n are not both even; 
0(-d - 1) © 3£>(-d) — > nO on P 2 such that 

0<A 1 <1, 

-2 + -(d 2 + 2d) < Ai < 1 - --^-(d 2 + 3d); 
n in 

mO(-d - 1) © 30(-l) — > nO on P 2 with m < a and 

1 Ti — 3 vt< — 6 

< Ai < , Ai(4m-3a + 3fe) < , Ai < . 

3a + to n mn 

Here a = (d + l)(d + 2)/2 and b = d(d + l)/2 and 3 f gcd(m, n); 
0(-d - 2) © 3£>(-d) — > nO on P 2 such that 

6 1 

19 <A2 < 3' 

A2 -2"2^ 2 

1 /d + 2\ 1 fd + 1 

A 2 < 1 

n 

1 /d + 2 



A 2 < - 

m£>(-d 3 ) © £>(-d 2 ) © O(-di) — S- nO on P r satisfying 

/ di - d 2 + r — 1 N 
r - 1 

1 - toAi - a 3 iAi + a 32 a 2 iAi 



to < 



a 2 iAi < A 2 < 



1 + a 32 

A 2 < 1 - toAi, 

, . 1 1 / M 



to + a 2 i to 7i + a 2 in V r 
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The precise meaning of Ai and A2 is revealed in section 2: they encode the character 
of G relative to which the sets of stable points have been defined. Beside the above, 
we have at (3.3) a more general criterion giving sufficient conditions, expressed in 
terms of linear algebra constants, under which morphisms of type (2,1) 

miO(-di) © m 2 0(-d 2 ) — > nO 

satisfy the compatibility conditions from [3] that lead to existence of geometric 
quotients. 

The paper is organized as follows: in section 2 we supply background material 
about quotients modulo nonreductive groups. As this is not part of mainstram 
Geometric Invariant Theory, we felt it necessary to reproduce the main results and 
definitions from [3]. Section 3 contains our criterion for morphisms of type (2,1). We 
apply this criterion in section 7. The remaining sections are indirect applications 
of (3.3), by which we mean applications of the method of proof, rather than the 
statement of (3.3). All polarizations we discuss are assumed to be nonsingular in a 
sense that is made precise in our very brief discussion of the Geometric Invariant 
Theory fan, cf. the beginning of section 3. 

2. Quotients Modulo Nonreductive Groups 

In this section we reproduce some notations and definitions and we quote the 
main result from [3]. We also quote two particular cases of King's Criterion of 
Semistability, as formulated in [3], which we will use in the subsequent sections. 

Fix a vector space V over k of dimension n + 1 . Drezet and Trautmann consider 
coherent algebraic sheaves £ and T on the projective space P" = F(V) having 
decompositions 

l<i<r l<l<s 

Here , N[ are vector spaces over k of dimensions wii, m. In [3] it is assumed that 
Si and Ti are simple sheaves, but for the purposes of this paper we will assume that 
they are line bundles: 

£i = 0(ei), ei<...<e r , Ti = 0(/j), h <...</,. 

The linear algebraic group Aut(£) x Aut(7 r ) acts by conjugation on the finite di- 
mensional vector space 

W = Hom(£, F). 

The subgroup of homothctics, which we identify with k* , acts trivially so, without 
losing any information, we can instead consider the action of the quotient 

G = Aut(£) x Aut(J")/fc*. 

If r > 1, or if s > 1, the group G is nonreductive, however, it contains the reductive 
subgroup 

G rcd = GL(Mi) x ... x GL(M r ) x GL(JVi) x . . . x GL(N s )/k*. 
We represent elements of G rc d by pairs (g, h), with 

g=( gi ,...,g r ), h= (h u ...,h s ), Sl £ GL(M t ), hj G GL(Nj). 
The characters of G re d are of the form 

X (g,h)= J] det( 52 r A -- [J det(^r 

l<i<r l<J<s 
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for integers Ai, . . . , A r , /ii, . . . , /U s . As \ must be trivial on the subgroup of homo- 
theties, we require the condition 



Clearly \ extends to a character of G, which we also denote by \- Drezet and 
Trautmann call a polarization the tuple 



They consider a semistability notion for G rc d depending on A. We quote below the 
definition. We point out that, relative to a certain linearization, this is the usual 
notion of semistability from Geometric Invariant Theory. This is made precise at 
lemma 3.4.1 in [3]. 

(2.1) Definition: Let A be a fixed polarization. A point ip G W is called 

(i) semistable with respect to G re d and A if there are an integer m > 1 and an 
algebraic function / on W satisfying f(g.w) — X m (</)/(w) for all g G G rc d 
and w G W, such that f(<p) ^ 0; 

(ii) stable with respect to G rc d and A if the isotropy group of ip in G rc d is finite 
and there is / as above, but with the additional property that the action 
of G re d on the set {w G W, f(w) ^ 0} has closed orbits. 

This definition is consistent because proportional tuples of integers give rise to the 
same sets of semistable (stable) points. Let T be the maximal torus of G rc d- Note 
that T is also a maximal torus in G. A point ip G W is semistable (stable) with 
respect to G rc d and A if and only if every point in its G re d-orbit is semistable 
(stable) with respect to T and the restriction of \ to T . Taking this equivalence as 
definition for semistability (stability) with respect to G we arrive at the following 
concept introduced by Drezet and Trautmann: 

(2.2) Definition: A point ip G W is called semistable (stable) with respect to G 
and A if every point in its G-orbit is semistable (stable) with respect to G rc d and A. 
We denote by W SS (G, A) and W S (G,A) the sets of semistable, respectively stable 
points in W. 

For checking semistability in concrete situations we need a criterion derived by 
A. King in [1] from Mumford's Numerical Criterion. We use its formulation from 
[3]. Below we quote only a particular case. Let us represent a point ip G W by a 
matrix 

{<Pli)i<l<s,i<i<r with (flu G Hom(Mi ® H* it JVj), H u = Hom(£j, 

A family of subspaces M[ C Mj, N[ C Ni will be called admissible if not all 
subspaces are zero and we do not have M[ = M i7 N[ = Ni for all i, I. 

(2.3) Proposition: A morphism ip G W is semistable (stable) with respect to G 
and A if and only if for each admissible family of subspaces M[ C M i} N[ C Ni, 
which satisfies 




KKr 



KKs 



A = (-Xi/d, -X r /d, /ii/d, . . .,n s /d). 



for all i, I 
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we have 

s r 

J2mdim(N{) > (>) Ai dim(M-). 
1=1 i=l 

Drezet and Trautmann embed the action of G 011W into the action of a reduc- 
tive group G on a finite dimensional vector space W . They introduce associated 
polarizations 

A = (ai, ...,a r ,/3i,.. .,&) 

and define sets of semistable and stable points W SS (G, A), respectively W S (G, A), 
as at (2.1). In general they determine the following relationships: 

(2.4) Proposition: Let ( : W — > W denote the embedding map. Then we have 
the inclusions 

C^W^iG,!)) c W SS (G,A) and (~ 1 (W S (G,A)) C W S (G,A). 

They also find sufficient conditions under which the reverse inclusions hold but, in 
this paper, we will not use them. The main result from [3] states that, if the sets 
of semistable (stable) points in W and W are compatible, then there are good or 
geometric quotients modulo G. We quote below the part that will be used in the 
sequel: 

(2.5) Proposition: If (~ 1 (W S (G, A)) = W S (G, A), then there exists a geometric 
quotient W s (G, A)/G which is a smooth, quasiprojective variety. 

Combining (2.4) and (2.5) one arrives at the following: 

(2.6) Proposition: If ((W S (G, A)) C W S (G,A), then there exists a geometric 
quotient W S (G,A)/G which is a smooth, quasiprojective variety. 

The goal of this paper is to give classes of examples of geometric quotients that 
result as applications of (2.6). We will avoid any discussion of properly semistable 
morphisms, i.e. morphisms which are semistable but not stable. Drezet and Traut- 
mann's theory works under certain a priori restrictions on the polarization A. First, 
it is noticed that the set of semistable (stable) points in W is nonempty only if 

Ai > (>)0 and m > (>)0 for all i, I. 
Other conditions can be found at 5.4.1 in [3]: W S (G, A) is not empty only if 

(2.7) a 2 >0 for (r,s) = (2,l), 
a 3 >0, Aipi<l for (r,s) = (3,l). 

We will describe the embedding only in the case (r, s) = (2,1). Let us write 

£ 1 = O(-di), £ 2 = 0(-d 2 ), Ti - O. 
The polarization A is a triple 

A = (— Ai, — A2, Hi), with Hi = — and m 1 A 1 +m 2 A 2 = l. 
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Recall that 

H n = Hom(fi, T x ) = S dl V\ H 12 = Hom(£ 2 , F x ) = S d2 V*. 
Consider also the spaces 

A 2 i =Hom(fi,f 2 ) - S dl - d2 V*, 
Pi = Hom(£i,£) = Mi © M 2 <g> A 2 i, 
P 2 = Hom(£ 2 ,£) =M 2 , 

W = Hom(P 2 <g> A 2 i,Pi) 8 Hom(Pi,ffn(8)iVi). 

Small case letters a = a 2 i, pi = mi +m 2 a 2 i, p 2 = m 2 denote the dimensions of the 
corresponding spaces. The group G acting by conjugation on W is 

G = GL(Pi) x GL(P 2 ) x GL(JVi)/jfe*, 

and the associated polarization is 

A = (— a\, — a 2 , /?i) with ax = Ai, a 2 = A 2 — aAi, /3i = Mi = — . 

m 

We represent morphisms in W as matrices ip — {ip' , <p"), 

<P = (<^ij)l<i<ni,l<j<mn Vij € Hu, 

= (Vij)l<i<ni,mi<j<mi+m 2 ) V'ij ^ ^12- 

We write i£>j for the j th column of tp. We consider the row vector 

X=[X X ■■■ X a ] 
with entries forming a basis of ^4 2 i. The p\ x p 2 -matrix with entries in ^4 2 i 



i = 






• 





X T 


• 








X T ■ 








• 


• X 1 



represents an element in Hom(P 2 <g> A 2 i,Pi). The n\ x pi-matrix with entries in 

7 (< /9 ) = [ tp 1 tp mi+1 X ■ ■ ■ tp mi+m2 X ] 

represents an element in Hom(Pi,Pn <g> Ni). We put ((p) = (£,7(1^)). It is clear 
that "{(hup) — hi-f(<p) for all h\ G GL(TVi). For this reason, when it comes to 
semistability considerations, we can, and we will assume that hi is the identity 
automorphism. 

We finish this section with the particular case of King's Criterion of Semistability 
that applies to W: 

(2.8) Proposition: A point (^,7) of W is semistable (stable) with respect to G 
and A if and only if for each admissible family of subspaces P[ C Pi , P 2 C P 2 , 
C N\ satisfying 

x(P' 2 ® A 21 ) C Pi', j{P[ ® Pi*i) C N[, 

we have 

/?i rfim(A^i) > (>) ai dim(P[) + a 2 dim(P 2 ). 
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3. A Criterion for Morphisms of Type (2,1) 

We fix integers d\ > d 2 > 0, we fix a vector space V of dimension r + 1 and we 
consider morphisms 

L P ={<p',v"):m 1 0{-d 1 )®m 2 0{-d 2 )^nO on P r = P(V). 

The polarization A is uniquely determined by Ai 6 [0, 1/mi]. The theory of the 
GIT-fan, as developed in [5] and other works, informs us that there are finitely 
many values = sq < si < . . . < s q = 1/mi such that when Ai varies in an 
interval (s K , s re +i) the set of semistable morphisms docs not change and each open 
interval (s K , s K +i) is maximal with this property. The intervals (s K , s K +i) are called 
chambers. The points s K are called singular values for Ai. Either Ai or A2 uniquely 
determine A, so we can talk of singular polarizations A, or singular values for A 2 . 

According to King's Criterion of Semistability (2.3), whenever the set of properly 
semistable morphisms with respect to A is nonempty, there is an equality 

s r 

2 W dim(JV/) = 2A i dim(Mf). 
1=1 i=i 

Those polarizations for which there is an equality as above, for some choice of 
subspaces N[ and will be called irregular, or wc may say that Ai or A 2 is 
irregular. There are situations in which all polarizations are irregular, but those 
situations will not be addressed in this paper. The other possibility for morphisms 
of type (2,1), which wc assume henceforth, is a finite set of irregular polarizations. 
From King's Criterion of Semistability we see that W S {G, A) depends only on the 
set of tuples of integers (ai, . . . , a r , b\, . . . , b s ), < ai < rrij, < h < nj, for which 
there is an inequality 



Let now A be a fixed regular polarization. If A is sufficiently close to A , meaning 
that Ai is sufficiently close to A?, then A is also regular and the sets of tuples of 
integers with the above property for A and A are the same. Thus, the sets of stable 
morphisms, which for regular polarizations coincide with the sets of semistable 
morphisms, are the same. In other words, any open interval bounded by two 
consecutive irregular values for Ai is contained in a chamber. Thus, all singular 
polarizations are irregular. The author does not know if the converse statement is 
also true, but he will give below an example in which the irregular polarizations 
are singular. These considerations also show that for A in a chamber we have 
W SS (G, A) = W S (G, A) because, if A happens to be irregular, we can perturb it 
slightly to a regular polarization in the same chamber. 

Given integers < K\ < mi and < «2 < we denote by l KlK2 the smallest 
integer satisfying 

; 

> K1A1 + K2A2 



n 

and we consider morphisms of the form <p KlK2 = (<p' KlK2 , ( Pk 1 k 2 )^ wriere 



0/„ 



0z K 



k1 k 2 ,m 2 -K 2 
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By 0;fc we denote the identically zero I x fc-matrix. According to King's Criterion 
(2.3), the morphism <p is semistable if and only if it is not equivalent to p KlK2 for 
any choice of K\ and k 2 . 



(3.1) Example: In the simplest case m\ = m 2 = 1 the irregular polarizations are 
of the form A = (n/n, 1 — n/n, 1/n), < n < n. The set of semistable morphisms 
may be empty for some polarizations, for instance, if n > k + d\m(S d2 V*) and 
n/n < Ai < (k + l)/n. We see this from the semistability conditions: ip is in 
W SS (G, A) if and only if ip is not equivalent to a morphism of the form 

* K +i,i 

The following two conditions are sufficient to guarantee the existence of semistable 
morphisms corresponding to all polarizations: 

n<( rJr< ^ 2 | and n< ( ^ ^ \ 

To see this we choose a nonzero linear form -0 e V* and a linear complement 
U C V* of the subspace generated by i[>. We choose linearly independent elements 

<pn,...,<Pni£S dl U and V12, • • ■ , V>n2 € S^V*. 

We put (fi2 = ipi2 ?A for 1 < i < n. We claim that ip is not equivalent to a 
matrix having a zero entry. Indeed, the entries from the second column are linearly 
independent, and no linear combination of entries from the second column can 
divide a linear combination of entries from the first column, because the former is 
divisible by ip, whereas the latter is not. 

Under the above conditions on n we see that the singular polarizations are pre- 
cisely the irregular ones. Indeed, for Ai = n/n there exist properly semistable 
morphisms: just choose ipn, . . . , ip K i linearly independent in S dl U, put ipn = for 
k + 1 < i < n and p i2 — V^^A as above. 



or 



On- K ,l 



We now turn to the embedding into the action of the reductive group. In order 
to apply the theory from section 2 we need to assume a priori that a 2 > 0, that is 
A 2 > a\i. According to (2.8), a point (£,7) G W is semistable if and only if 

(£,7) 00 (6i,7»0 witn ^ > +iot2- 



Here jik are matrices of the form 





Ilk 



Because of the special form of £ we must have k < m\ + ai. We conclude that, in 
order to ensure the semistability of (£,7), it is enough to require the condition 

7 1° Ilk with — > fcAi + ia 2 and k < m\ + ai. 
n 

Actually, it is enough to require 

(3.2) 7 ^ "/ik with — > fcAi + ia 2 , mi + (i — l)a < k < m\ + ia, < i < m 2 . 
n 

Before we state the main result of this section we introduce some linear algebra 
constants very similar in definition to the constants c; and di from [3]. Given 
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integers 1 < i < m 2 — 1 and 1 < j < m 2 a we denote by k(i,j) the maximal 
dimension of a vector space U C M 2 <8> i?i2 which is not contained in M 2 ® H12 
for any subspace M 2 C M 2 of dimension i and for which there is a subspace in 
M| © A 2 i of dimension at least j orthogonal to [/ under the canonical bilinear map 

(M 2 <g> H12) x (M 2 * ® A 2 i) — > Hu. 

Given an integer 2 < i < m 2 and a vector space M of dimension z we let k(i) be the 
maximal dimension of a subspace [/ C M® f/12 which is not contained in M' © H12 
for any proper subspace M' C M, and for which there is a nonzero subspace in 
M* © A21, orthogonal under the canonical bilinear map 

(M © iJi 2 ) x (M* © A 2 i) — > i?n. 

(3.3) Claim: Let &e t/ie space of morphisms 

Lp : miO(-di) ©m 2 0(-d 2 ) — >u0 on P r = P(y). 

Assume that m\ < a — dim(S dl ~ d2 V*). Let A be a nonsingular polarization satis- 
fying the following conditions: 

A 2 > a\i, 

in\ 2 > nmiXi + k(i, m 2 a — ia — mi) for 1 < i < m 2 — 1, 

mA 2 > nmiAi + k(i) for 2 < i < m 2 , 

nAi + mA 2 > fc(i, m 2 a — ia — a + 1) for 1 < z < m 2 — 1. 

T/ien W ss (G, A) admits a geometric quotient modulo G, which is a quasiprojective 
variety. 

Proof: Let ip be in W SS (G, A). According to (2.6), we need to show that (£, 7(y>)) is 
stable. Perturbing slightly A we can ensure that A is a nonsingular polarization and, 
at the same time, that W SS (G, A) has not changed. Thus W SS (G, A) = W S (G, A) 
and it is enough to show that 7(1^3) satisfies condition (3.2). 

We will argue by contradiction. Assume that j(ip) is equivalent to some jik with 

— > kX\ + ia 2 , mi + (i — l)a < k < m\ + ia, < i < m 2 . 
n 

Let ip, -0', "0" denote the truncated matrices consisting of the first I rows of ip, ip' , 
</?". In view of the comments before (2.8), we may assume that the vector subspace 
in Mi <E> Hu © M 2 <E> i/ 12 spanned by the rows of ip is orthogonal to a subspace, 
denoted Ker(ip), inside M-j" © M| ©^4 2 i, of dimension at least p\ — k. Orthogonality 
here is understood under the canonical bilinear map 

(Mi <g> Hu © M 2 O H 12 ) x (Mi © M 2 © A 2i ) — > ff n . 

By analogy, considering the pairings 

(Mi © Hu) x M* — > Hn and (M 2 © #12) x (M 2 © A 2 i) — > H n , 

we define the orthogonal subspaces Kcr(i//) and Ker(tp"). The dimensions of the 
corresponding spaces are denoted by ker(0), ker(0'), ker(0"). 

Case 2 = 0, 1 < k < mi. As ker(-0") > m 2 a — k > (m 2 — l)a we see that Ker(-0") 
intersects nontrivially every copy of A 2 i inside M| © A21 — k m2 © A21. Thus 
ip" = 0. Replacing possibly ip with an equivalent morphism we may assume that 
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ker(i£>') > mi — fc. We obtain that tp is equivalent to tpko, which contradicts the 
semistability of tp. 

Case i — 1, mi < fc < a. Again ?//' = 0. As Z/n > miAi + a 2 > miAi we arrive at 
the contradiction </? ~ <p mi ,o- 

Case i = 1, a < fc < mi + o. If i/>" ~ Vi , then ker(^") = (m 2 — l)a hence, after 
possibly replacing ip with an equivalent morphism, we may assume that 

ker(?/>') >mi + m 2 a — fc — (m 2 — l)a = mi + a — fc. 

Moreover, from Z/n > fcAi + a 2 = (fc — a)Ai +A 2 we get Z > h-a.i- Thus <p <~ <Pk-a,i, 
contradiction. 

If ■)/)", then the rows of tp" span a space of dimension at most 
fc(l, m 2 a — fc) < fc(l, m 2 a — a — mi). 
From the hypotheses of the claim we have 

Z > nfcAi + n«2 > naAi + na 2 = nA 2 > nui\\\ + fc(l, m 2 a — fc) 
forcing I > l mi ,o + fc(l,m 2 a — fc). We conclude that tp <~ <£> TOll o 5 contradiction. 

Case z > 2, mi + (i — l)a < k < ia. If i//' ~ V'f-u then, taking into account the 
inequalities 

— > miAi + (i — l)aAi + ia 2 > miAi + (i — 1)A 2 , 
n 

we obtain the contradiction tp ~ <^ mi ,,-i. Assume now that V" ^ V'f-i- The rows 
of ip" must span a vector space of dimension at most k(i — 1, m 2 a — fc). From the 
hypotheses of the claim we have the inequalities 

I > nkXi + nia 2 > n(mi + (i — l)a + l)Ai + nia 2 

> nmiAi + nAi + n(i — 1)A 2 

> nmiAi + k(i — 1, m 2 a — ia + 1) 

> nmiAi + fc(i — 1, m 2 a — fc). 

From the above we get I > l mi ,o + k(i — 1, m 2 a — fc), leading to the contradiction 

Case i > 2, ia < k < mi + ia. As above, if ~ V'f-i we get a contradiction. 
Assume now that ~ V'f an d V 1 " 1,0 V'f-i- If ker(V>") = (m 2 — i)a, then, replacing 
possibly tp with an equivalent morphism, we may assume that 

kev(tp') > mi + m 2 a — fc — (m 2 — i)a = mi + ia — fc. 

Moreover, from the inequalities l/n > k\\+ia 2 = (k — ia)Xi+i\ 2 we get I > h-ia.i- 
These lead to the contradiction <p <~ tpk-ia,i- 

If ker(?/>") > (m 2 — i)a, then the rows of tp" span a space of dimension at most 
k(i). From the hypotheses of the claim we have the inequalities 

I > nkXi + nia 2 > niaXi + nia 2 = ni\ 2 > nmiAi + k(i). 

Thus I > l mi .o + k(i) leading to the contradiction tp ~ ip mi ,o- 
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Finally, assume that ip" °° ip'/- Then the rows of ip" span a space of dimension 
at most k(i, m 2 a — k). From the hypotheses of the claim we have the inequalities 

I > nkXi + nia 2 > niaXi + nia 2 = niX 2 

> nm\X\ + k(i, m2(i — ia — mi) 

> nmiAi + k(i, m 2 a — k). 

From the above we get I > l mi ,o + k(i, m 2 a — k), hence the contradiction ip ~ ip mi ,o- 

The disadvantage of (3.3) is that the linear algebra constants k(i,j) and k(i) are 
difficult to compute in general. We carry out the computations only for a particular 
kind of morphisms in section 7. Sections 4, 5, 6 and 8 are not direct applications 
of (3.3): in the above proof a lot of information is lost in the course of estimating 
I. We can get more precise statements by examining each case separately, yet the 
arguments will be just reworkings of the arguments from the proof of (3.3) 



4. Morphisms of the Form mO(-di) © 20(-d 2 ) — > nO 

We fix integers d\ > d 2 > 0, we fix a vector space V of dimension r + 1 and we 
consider morphisms 



<p = (<p',<p") :mO{-d 1 )®20(-d 2 ) — > nO on 



P(V). 



The singular values for Ai are among those values for which there are integers 
< k < n and < p < m such that 



= pXi 



or 



= pAi+A 2 or -=pAi+2A 2 . 



n 



If both m and n are even we can choose k = n/2, p — m/2 and the second equation 
will be satisfied for all Ai, in other words all polarizations will be irregular. To avoid 
this, we will assume in the sequel that either m or n is odd. Under this assumption 
the singular values for Ai are among the numbers n/pn, < k <n, 1 < p < m. 

The semistability conditions in the special case m = 1 read as follows: for 
n/n < Ai < (K + l)/n the morphism ip is semistable if and only if ip is not equivalent 
to a matrix having one of the following forms: 







' 0; 2 ,2 * " 








' 0j 4 ,l * " 




, f2 = 




, ¥3 = 




, <f4 = 











where 

h = k + 1, 



h = 



n — k + 1 



n + k 



+ 1, 



The set of semistable morphisms may be empty, for instance when n > k + 
2 dim(S d2 V*). However, the following three conditions are enough to guarantee 
the existence of semistable morphisms corresponding to all chambers (we are still 
in the case m = 1): 



(4.1) n < dim(S ,d2 - 1 U*)+/ 3 -l, n < 2dim(S* , - 1 V*)+/i-l ) n < dim{S dl U). 
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We refer to (3.1) for the meaning of U. Explicitly, the above conditions take the 
form 

n < 2 ^ r ^ 2 + k — 1 when n + k is odd, 

n<2[ r ^^ 2 ^ I + k when n + k is even, 



n < 



r 

r + di - 1 
r - 1 



We are not able to say precisely what are the singular values for Ai in general, 
however, in the special case m = 1, and under the assumption (4.1), one can see as 
at (3.1) that the singular values are n/n, < k < n. 

We now turn to the embedding into the action of the reductive group. In order 
to apply the theory from section 2 we need to assume a priori that a 2 > 0, that is 
Ai < l/(2a + to). According to (3.2), a point (£, 7) g W is semistable if 7 ^ 7;^ 

with — > k\i and < k < to, 

n 

or with — > k\i + a 2 and m+l<fc<a + m, 

n 

or with — > k\i + 2a 2 and a + m+ l<fc<2a + m— 1. 

n 

(4.2) Claim: Let m and n be positive integers at least one of which is odd. Let W 
be the space of morphisms 

<p:mO(-d 1 )®20(-d 2 )^nO on P r =P(y). 

Let < Ai < l/(2a + m) be a nonsingular value. Assume that either the conditions 

,., fr-l + d 1 -d 2 \ , n- dimiS^V*) 

I to < and Ai < — — 

\ r — 1 ) (a + TO — l)n 

or the conditions 

.... fr + d±-d 2 \ n-2dim(S d i- 1 V*) 

II to < and Ai < 

w V r J 3mn 

are satisfied. Then the set of semistable morphisms admits a geometric quotient 
W SS (G, A)/G, which is a quasiprojective variety. 

Proof: Let ip be in W SS (G, A). According to (2.6), we need to show that (£,7(<p)) 
is semistable. We argue by contradiction. Assume that 7(9?) ~ jik with l/n > k\i 
and < k < to. Let ip — (tp' 7 ip") denote the truncated matrix consisting of the 
first I rows of ip. By assumption -0 has kernel inside k m ® S dl ~ d2 V* © S dl ~ d2 V* of 
dimension at least 2a+m— k which is greater than a+m because, by hypothesis, to < 
a. This shows that the kernel of ip intersects each copy of S dl ~ d ' 2 V* nontrivially, 
forcing ip" = 0. Moreover, there are at least to — k linearly independent elements 
in the kernel of ip' viewed as a subspace of k m . We get ip ~ <Pfc,o, which contradicts 
the semistability of ip. 

Assume now that ~/(ip) ~ "/ik with l/n > fcAi + ct 2 and TO + l<fc<a + TO. Note 
that automatically / > l m .o, thus excluding those 7^ with k < a because, as we 
saw above, the condition k < a forces ip" = 0, yielding the contradiction ip ~ <p m ,o- 



ON SOME QUOTIENTS MODULO NONREDUCTIVE GROUPS 



13 



Assume that a < k < a + m. We have ker(-0) > 2a + m — k > m, which forces 
ker(ip") > 1. Let (/, g) be a nonzero vector of Kei{ip") regarded as a subspace of 
S dl ~ d2 V* ®S dl ~ d2 V* . Assume that /, g are linearly dependent. Replacing possibly 
ip with an equivalent morphism we may assume that g — 0, / ^ so the first column 
of ip" is zero. The second column of ip" is not zero because if (p m fi. Now we 
have kei(ip") = a and, replacing possibly p with an equivalent morphism, we may 
assume that ker(ip r ) > a + m — k. As l/n > (k — a)Xi + X 2 we arrive at p <~ Pk-a,i, 
contradiction. 

Assume now that /, g are linearly independent. We write / = hfi, g = hgi with 
/i, gi relatively prime and max{0,di — 2d 2 } < d = dcg(/i) < d\ — d 2 . The rows of 
ip" are of the form (—giu, fiu), hence they are vectors in a space of dimension equal 
to dim(S 2d2+d ~ dl V*). As ip" ^ we see that the kernel of ip" consists of vectors 
of the form {vf u vgi), hence ker(>") = dim(5 d y*). Writing b = dim(5 dl - d2 - 1 V r *) 
we have 

m > kcr(V/) > 2a + m - k - dim{S d V*) >2a + m-k-b 

hence 

b + k >2a so b + a + m >2a giving m> a — b. 

This contradicts hypothesis (i) from the statement of the claim. Under hypothesis 
(ii), we would get a contradiction if we could show that the inequality l/n > k\i+a 2 
implies the inequality 

/ > l m fi + dim(S 2d2+d - dl V*). 

Indeed, we would arrive at the contradiction ip ~ ip m ,o- Thus, for all a < k < a + m 
and d we need the inequality 

nk\ 1 + na 2 > mn\ 1 + dim(S 2d2+d - dl V*). 

This would follow from the inequality 

naXi + na 2 > mnXi + dim(S d2 ~ 1 V*). 

But the above is equivalent to the condition on Ai from hypothesis (ii). 

We are left to examine the situation "f(ip) ~ "/ik with l/n > fcAi + 2a 2 and 
a + m + l<k<2a + m — 1. As k\i + 2a 2 > (m + l)Ai + X 2 + a 2 , we see that 
I > l m ,i- If k < 2a then, as above, we get ker(?/>") > 1. Let /, g be as above. 
If /, g are linearly dependent, we get the contradiction ip ~ p m .i- If /, 9 are 
linearly independent, we get a contradiction as above under hypothesis (ii). Under 
hypothesis (i) we would also get a contradiction if we could show that the inequality 
l/n > kXi + 2a 2 implies the inequality 

I > l m ,o + dim(S 2d2+d - dl V*). 

Thus we need the estimate 

n(a + m+ l)Ai + 2na 2 > mnX x + dim(5" i2_1 U*). 

Using the relations a 2 = X 2 — aXi and 2X 2 = 1 — mAi we see that the above is 
equivalent to the estimate on Ai from hypothesis (i). 

Finally, assume that 2a < k < 2a + m — 1. If ker(ip") > 1 we get a contradiction 
as above. If ker^") = 0, then ker(^') > 2a + m-k. As l/n > (A:-2a)Ai + 2A 2 we 
arrive at the contradiction ip ~ Pk-2a.2- This finishes the proof of the claim. 



14 



MARIO MAICAN 



According to corollary 7.2.2 in [3], if a 2 > and if A2 > ct2iCi(2)/n, then the 
conclusion of (4.2) is true. The constant Ci(2) can be computed as in the proof of 
lemma 9.1.2 (loc. cit.) One has 

dim(S d2 V) 

Cli - ' ~ dim(S^-^V)' 

Thus, according to Drezet and Trautmann, the conclusion of claim (4.2) holds if 

1 (r + d 2 \ , , If 2 (r + d 2 

A 2 > - , that is Ai < — 1 - - 

n\rj m \ n \ r 

Our result is not contained in Drezet and Trautmann's result. 

In the special case m = 1, r > 2 the estimates on m from (4.2) (i) and (ii) are 
automatically fulfilled, so we have: 

(4.3) Claim: Let W be the space of morphisms 

(p :0{-d l )®20{-d 2 ) — > nO on P r , r > 2. 

Let < Ai < l/(2a +1) be a nonsingular value satisfying one of the following two 
conditions: 

a an\ r J 6 in\ r 

Then the set of semistable morphisms admits a geometric quotient modulo G, which 
is a quasiprojective variety. 

At the end of this section we would like to spell out a simple case in which we 
can say for sure that the above claim is nonvacuous, that is, in which we know 
that W SS (G, A) is not empty. The case of the left-most chamber < Ai < 1/n is 
completely understood, cf. lemma 9.3.1 in [3]. Let us take 1/n < Ai < 2/n and 

(4.4) n = 2^ + d *~ 1 ^j +1, d 1 <2d 2 -2. 

Condition (4.1) is satisfied for k = 1 because n + n is even. The conditions on Ai 
from (4.3) reduce to < A x < l/(2a + 1). We have 1/n < l/(2a + 1) because 
of the second inequality in (4.4). We conclude that claim (4.3) is nonvacuous for 
morphisms satisfying (4.4). 

5. Morphisms of the Form 0(-d - 1) © 30(-d) — > nO 

We fix an integer d > 0, we fix a vector space V over k of dimension 3 and we 
consider morphisms 

<p= (<p',<p") : 0(-d-l)®30(-d) nO on P 2 = F(V). 

Keeping the notations from section 2 we have: 

a = 3, mi = 1, m 2 = 3, p\ = 10, a 2 = X 2 — 3Ai = - — ^— ^. 

The singular values for Ai are among those values for which there are integers 
< k < n and < p < 3 such that 

- = p\ 2 or -=\ 1 +p\ 2 . 

n n 
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" * 0*1,3 " 


, ¥>2 = 


' 0; 2 , 3 * " 


» ¥>3 = 


' * 0/ 3j2 " 






* * 


' 0(4,2 * " 


» ¥>5 = 




, <P6 = 


' 0/ 6 ,i * " 









Using the relation Ai + 3A2 = 1 we sec that the singular values for Ai are among 
the numbers n/2n, < k < 2n. 

We write tp as a matrix ((pij)i<i< n , i<j<4 with p>n G S d+1 V* while ipij £ S d V* 
for j = 2, 3, 4. The morphism ip is semistable if and only if it is not equivalent to 
a matrix having one of the following forms: 



<Pi 



lf4 = 

Here each ipi has a zero submatrix with k rows. The integers I are the smallest 
integers satisfying 

->A l7 ^>A 2 , ^>A 1+ A 2 , ^>2A 2 , ^>A 1+ 2A 2 , l -° > 3A 2 . 
n n n n n n 

In order to apply the theory from section 2 we need to have a 2 > 0, that is we 

need to assume a priori that Ai < 1/10. According to (3.2), in order to show that 

(£, 7) is semistable, it is enough to show that 7 00 -y lk 

— > Ai and k = 1, 

n 

' > fcAi + a 2 and k = 2, 3, 4, 



with 
or with 
or with 
or with 



n 

n 
I 



> fcAi + 2a 2 



> fcAi + 3a 2 



and k = 5, 6, 7, 
and fc = 8, 9. 



(5.1) Claim: Let W be the space of morphisms of sheaves on P 2 of the form 

if : 0(-d - 1) © 30(-d) — ► nO. 
TTierc for any nonsingular value for Xi satisfying the conditions 

0<A,<±. 

Ai < l-^-{d 2 + 3d), 
An 

x^-l + ly + d), 

Ai > -2+ -(rf 2 + 2d) 
n 

the set of semistable morphisms admits a geometric quotient modulo G, which is a 
quasiprojective variety. 

Notice that for d and n large, and for n of order 3d 2 / 2 the last four inequalities 
follow from the first. 



Proof: Let ip be in W SS (G, A). According to (2.6), we need to show that (£, 7(99)) is 
semistable. We argue by contradiction. Assume that y(ip) ~ jik with l/n > Ai and 
k = 1, 2. Note that l>l\. Let ^ = (?//, ?/>") denote the truncated matrix consisting 
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of the first / rows of p. By assumption ip has kernel inside k V* V* V* of 
dimension at least 10 — k. For dimension reasons Ker(ip) intersects each copy of V* 
nontrivially, forcing ip" = 0. We get (p ~ <p\, which contradicts the semistability of 
ip. The same argument also works in the case k = 3 and kev(ip") = 7. 

Assume now that k — 3, l/n > 3Ai + a2 = A2 and ker(-0") = 6. Replacing ip with 
an equivalent morphism we may assume that ip' — 0. From remark (5.2) below we 
see that two columns of ip" vanish. As I > l 2 we get <p ~ <p2, contradiction. 

Assume that k = 4 and l/n> 4Ai + a 2 = Ai + A 2 . As ker(?/>") > 5 and I > I3, we 
can apply (5.2) and we get the contradiction ip ~ ip$. The same argument works if 
k = 5 and kcr(^") > 5. 

Assume now that k = 5, l/n > 5Ai + 2a 2 = (2 — 5Ai)/3 and that ker(^") = 4. 
The matrix has the form given at remark (5.3) below, so its rows are elements 
in a vector space of dimension at most equal to the dimension of S d ~ 1 V* . If we 
could show that I > dim(S d ~ 1 V*), then we would conclude that ip has a zero row, 
which would be a contradiction. Thus we need the inequality 



But this is equivalent to the second condition on Ai from the statement of the claim. 

Assume that k = 6 and l/n > 6A1 + 2a 2 = 2A 2 . Thus I > I4 and, from 
the above inequality, I > h + dim(S d ~ 1 V*). If ker(V>") = 4, then ip" has the 
form given at remark (5.3) below and we arrive at the contradiction ip <~ ip\. If 
ker('0") = 3, then we may assume that ip 1 = 0. Let 77 be a 3 x 3-matrix with 
entries in V* whose columns arc linearly independent vectors in Ker(i/>"). Each 
column of 77 must contain at least two linearly independent elements, otherwise we 
get the contradiction ip ~ ip^. Also, each row of r\ must contain at least two linearly 
independent elements. Indeed, if 



and we conclude that the first two columns of ip" are zero, again a contradiction. 

Thus 77 satisfies the hypotheses of remark (5.4) from below. We must have 77 = 771 
or 7] = r}2 ■ In the first case the rows of ip" are elements in a vector space of dimension 
at most equal to the dimension of S d ~ 1 V*. As I > dim(5" i_1 V r *), we conclude that 
ip has a zero row, contradiction. 

Assume now that 77 = r\ 2 . From remark (5.5) we know that the rows of ip" are 
elements in a vector space of dimension at most (d 2 +3d)/2. The third condition on 
Ai from the statement of the claim is equivalent to the inequality 2nA2 > (d 2 +3d) /2. 
This shows that I > (d 2 + 3d) /2, forcing ip to have a zero row, again a contradiction. 

The case k = 7 follows from the last two conditions on Ai from the claim and 
the cases k = 8 and k = 9 are analogous. This finishes the proof of the claim. 




77 = 



Ml u 2 u 3 
vi v 2 v 3 
w 3 



with u\, vi linearly independent, u 2 



v 2 linearly independent, then 



Ml u 2 
Vi v 2 



^0 



(5.2) Remark: Let ip be an I x 3-matrix with entries in S d V* . We consider the 
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space of 3 x 1-matrices rj with entries in V* such that ipr) = and we call it Ker(-0). 
Let kei(ip) denote its dimension. If kcr(tp) > 5, then two columns of i\> must vanish. 

(5.3) Remark: With the notations from (5.2), if ker(-0) = 4, then ip has the form 

" u/i vh ' 

_ ufi vfi 

with u, v G V* linearly independent one-forms and fi G S"^ 1 ^* for 1 < i < I. 
Next we quote 5.6 from [4]: 

(5.4) Remark: Let r\ be a 3 x 3-matrix with entries in V*. Assume that det(^) = 
and that rj is not equivalent, modulo elementary operations on rows or on columns, 
to a matrix having a zero row, or having a zero column, or having a zero 2x2- 
submatrix. Then rj is equivalent to one of the following matrices: 



X 


Y 







' X 


Y 


Z 


z 





Y 


or i] 2 = 


Y 


aiX + a 2 Y 


CI3X + C14Y + a^Z 





-z 


X 




Z 


aeX + a^Y + a$Z 


a 9 X + a w Z 



with nonzero constants 01, . . . , aio in the ground field k. Here {X, Y, Z} is a basis 



(5.5) Remark: The kernel of j) 2 inside S d V* ffi S d V* (B S d V* is of dimension at 
most {d 2 + M)/2. 

Proof: The elements of Kcr(77 2 ) are of the form 



with (/, g, h) determined modulo multiples of (Z, —Y, X). Without loss of generality 
we may assume that h depends only on Y and Z. We have 

= f(-Y 2 + ai X 2 + a 2 XY) + g(-YZ + a 6 X 2 + a 7 XY + a s XZ) 
+h(- ai XZ - a 2 YZ + a 6 XY + a 7 Y 2 + a s YZ). 

This shows that / is uniquely determined by g and h. Hence Ker(ry 2 ) is of dimension 
at most 



6. Morphisms OF the Form mO(-d - 1) © 3£>(-l) — > nO 
We fix an integer d > 0, we fix a vector space V of dimension 3, and we consider 



We write b = dim(5 d 1 V*) and, using the notations from section 2, we have 



of V*. 



f(-Y, X, 0) + g(-Z, 0, X) + h(0, -Z, Y) 




morphisms 



V = {if', ip") : mO(-d - 1) ® 30(-l) 



nO on P 2 = F(V). 



mi = m, m 2 — 3, pi = 3a + to, a 2 = X 2 — a\i 



1 — (3a + m)Ai 
3 
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The singular values for Ai are among those values for which there are integers 
< k < n, < p < m and < q < 3 such that n/n = pXi + qX%. If both m 
and n are divisible by 3 we can take k = n/3, p = m/3, q = 1 and we see that all 
values for Ai are irregular. If either to or n is not divisible by 3, which will be our 
assumption in the sequel, then, using the relation mX\ + 3X 2 = 1, we see that the 
singular values for Ai are among the numbers n/pn with < n < 2n, 1 < p < 2m. 

In order to apply the theory from section 2 we need to assume a priori that 
a 2 > 0, that is Ai < l/(3a + m). According to (3.2), in order to show that (£,7) is 
semistable, it is enough to show that 7 j lk 





with 


n 


> fcAi 




and 


< k < to, 


or 


with 


I 

n 


> fcAi 


+ a 2 


and 


m+l<fc<a+TO, 


or 


with 


I 

n 


> fcAi 


+ 2a 2 


and 


a + TO + l<fc<2a + m, 


or 


with 


I 


> fcAi 


+ 3a 2 


and 


2a + TO + l<fc<3a + TO-l 



n 



(6.1) Claim: Let m and n be positive integers one of which is not divisible by 3. 
Let W be the space of morphisms 

ip : mO(-d- 1) 8 30(-l) — > nO on P 2 = P(V). 

Assume that m < a. Then for any nonsingular value \\ satisfying 



< Ai < 



1 



Ai(4TO-3a + 36) < 



n — 3 



Ai < 



n — 6 



3a + m ' n mn 

the set of semistable morphisms admits a geometric quotient W SS (G,A)/G which 
is a quasiprojective variety. 

Proof: Let ip be in W SS (G 1 A). The first part of the proof proceeds as at (5.1). So 
let us assume that 7(v?) ~ "fik with l/n > fcAi + a 2 and a < k < a + to. We have 
ker(tp") > a + 1. Let r be the dimension of the vector space spanned by the rows 
of ip". According to (5.2) we would have ker(-0") < a if r > 5, which is not the 
case. According to (5.3) we would have ker(ip") < dim(S d ~ 1 V*) if r = 4, which is 
not the case either. Thus r < 3. 

Assume that r = 3. Let i] be a 3 x 3- matrix with entries in V* formed from 
three linearly independent rows of ip". We cannot have 





^1 

Wl 



u 
v 2 * 
w 2 * 



with 



vi v 2 
Wi w 2 



^0 



because this would lead to ker(tp") < ker(ry) < a. Since det(r/) = 0, we can apply 
(5.4) to deduce that 77 = 771, 77 = r/ 2 or that r\ is equivalent to a matrix having 
a zero column. The last case leads to the contradiction ip ~ <pk~ a ,i- If V = T)i, 
we get ker(r]) < dim(S d ~ 1 V*). If 77 = 772, we get, in view of (5.5), the inequality 
ker(?7) < (d 2 + 3d)/2 = a — 1. Both of them are contrary to ker(?/>") > a + 1. 

Assume now that r = 2. Let 77 be a 2 x 3- matrix formed from two linearly 
independent rows of ip". If 77 had two zero columns we would arrive as before at 
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the contradiction ip ~ tpk-a,i- Moreover, 77 has two linearly independent elements 
in each row, otherwise we would get ker(?//') < a. If 

_ * * u 
^ v w 

with ti^O, then Ker(i/>") consists of matrices of the form 



wfi ■ ■ 


■ -wf. 


vfi ■ ■ 


■ vl 


9i ■■ 


9. 



with each gj uniquely determined by fj. This shows that ker(^") < dim(S d ~ 1 V*) 7 
contradiction. We conclude that 77 is not equivalent to a matrix having a zero entry, 
i.e. 77 satisfies the hypothesis of remark (6.2) from below. According to remark (6.3) 
we again arrive at a contradiction: kcr(-0") < a — 1. 

The case r = 1 and the remaining possibilities for 7(93) are dealt with in a similar 
manner. This finishes the proof of the claim. 

(6.2) Remark: Let 77 be a 2 x 3-matrix with entries in V*. Assume that, under 
the canonical action of GL(2) x GL(3), 77 is not equivalent to a matrix having a 
zero entry. Then 77 is equivalent to one of the following two matrices: 

_ [ X Y Z 
m ~[Y b 1 X + b 2 Y + b 3 Z c 2 Y + c 3 Z 

with 61, C2, C3 ^ 0, or 

_ [ X Y Z 
m ~[ Y b 2 Y + b 3 Z Cl X + c 2 Y + c 3 Z 

with 63, ci ^ 0. Here {X, Y, Z} is a basis of V*. 

Repeating the arguments from the proof of (5.5) we can estimate the dimensions 
of the kernels for the above matrices: 

(6.3) Remark: The kernels of 773 and 774 inside S d V* ® S d V* © S d V* arc of di- 
mension at most (d 2 + 3d)/2. For d = 2 the kernel is of dimension at most 4. 



Proof: We need to do only the case d = 2. Keeping the notations from (5.5) we 
have, say in the case rj = 773, the relation 

(-YZ + c 2 XY + c 3 XZ)g + {-b x XZ - b 2 YZ - b 3 Z 2 + c 2 Y 2 + c 3 YZ)h 
■' ~ —Y 2 + biX 2 + b 2 XY + b 3 XZ 

The kernel of 773 can thus be parametrized by five parameters in the ground field k, 
namely the coefficients of g and h, because we can assume that the coefficient of X 
in h is zero. The requirement that the above ratio be a polynomial gives algebraic 
conditions in the space of parameters. For g = Y, h = Y the numerator is not divis- 
ible by the denominator because no monomial of the numerator is divisible by X 2 . 
Thus the algebraic conditions are nontrivial, i.e. the kernel of 773 is parametrized 
by a proper subvariety of k 5 and, as such, it has dimension at most 4. 

The argument can be repeated for the kernel of 774: choose / = Y, h = Y and 
there will be no possibility for g. 
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In the simplest nontrivial case l/2mn < Ai < l/(2m — l)n the conditions from 
the claim take the form 

3a 1 3(a-b) 

— + -<n, 5<n+^- '-, 7<n. 

2m 2 2m 

The following conditions are also necessary to ensure the nonemptyness of the set 
of semistable points: 

n < lm,o + 9, n<l m .i+6, n<l m ^ + '-i. 

Here Z m , = 1 while l m .\, l m .2 are the smallest integers satisfying 

n + 1 4n + l 

tm,l > — 7, — j 'm.2 > ^ • 

3 

These three conditions on n are satisfied for n < 9. For instance, if m = [a/2], all 
the above conditions on n reduce to n € {7, 8, 9}. It is clear that for d sufficiently 
large the set of semistable morphisms is not empty. 



7. Morphisms of the Form 0(-d - 2) © 3£>(-d) — > nO 

We fix an integer d > 0, we fix a vector space over fc of dimension 3 and we 
consider morphisms 

<p= (<p',(p") : - 2) © 3£>(-d) -^nO on P 2 = P(V). 

Here mi = 1, 7712 = 3, a = 6 and the singular values for Ai are as in section 5. The 
morphism ip is semistable if and only if it is not equivalent to one of the morphisms 
of the form ipi,...,<p 6 from section 5. From this we see that W SS (G, A) is nonempty 
if and only if the following conditions are satisfied: 



n < h + 3 
n < l 3 + 2 
n < l 5 + 
n<l 2 + 
n < l 4 + 
n < l 6 + 



d + 2 

d + 2 
d + 2 
d + 4\ „fd + 2 



2 J \ 2 

d + A\ fd + 2 



2 J \ 2 
2 
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Taking into account the definitions of h, . . . , l& from section 5, the above conditions 
can be rewritten as 



^ ^ o ( d + 2 
n < nXi + 3 I ^ 

n < nXi + nX 2 + 2 

n < nXi + 2nX 2 + 



d + 2 
d + 2 



(7.1) -SnA^^ 4 )^^ 2 

„ o , + A (d + 2 

n < 2nA 2 + 

n < 3nA2 + 



2 ) V 2 
2 



Taking into account the relation Ai + 3A 2 = 1, the first three conditions are equiv- 
alent to 

The conditions on A from (3.3) for the existence of the quotient modulo G read as 
follows: 



(7.3) 



A 2 


> 


6A1, 


T1X2 


> 


nXi + 11) 


2nX 2 


> 


nAi + fc(2,5), 


2nX 2 


> 


nAi + fc(2), 


3nX 2 


> 


nAi + fc(3), 


nXi + nX 2 


> 


fc(l,7), 


nXi + 2nX 2 


> 


fc(2,l). 



Next we will compute the linear algebra constants k(i,j) and fc(i) using elemen- 
tary operations with matrices of homogeneous polynomials. We do only the most 
laborious case: 



(7.4) Claim: fc(2,5) - dimiS^V*) = 



d + l 
2 



Proof: In the sequel a will be a matrix with 3 columns and entries in S V* having 
linearly independent rows and linearly independent columns. Also, j3 will be a 
3 x 5-matrix with entries in S 2 V* having linearly independent columns, such that 
afi = 0. The constant k(2, 5) is the maximal number of rows that a could have for 
all choices of a and (3. 
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Let us fix a basis {X,Y,Z} of V* and a basis {ui, 
consider the matrices 



j} of S^V*. We 



a = 



[ X Y Z ] and /3„ = 



x 2 o 

X 2 




IF 



-Y 2 
XY 




-yz 
o 

XY 



The choice a = ao, = 0o shows that fc(2,5) > q. We will prove the converse 
inequality using induction on d, but before that we need to make a few observations 
about a and j3. Firstly, /3 cannot be equivalent to a matrix having two zeros on a 
column, otherwise the columns of a would be linearly dependent. Secondly, [3 must 
have linearly independent rows. If, say, the third row of [3 is zero, then the matrix 
/?i made of the first two rows of (3 has all maximal minors equal to 0. As [3\ has 
linearly independent columns, we see that (3i is forced to have linearly dependent 
rows, contradicting observation one. Thirdly, a cannot be equivalent to a matrix 
having a zero entry. If, say 0:13 = 0, then all maximal minors of (3\ are zero, forcing 
(3\ to have linearly dependent columns or linearly dependent rows. This contradicts 
observation one, respectively observation two. 

To begin the induction assume that d = 1. We have to show that a cannot 
have more than one row. Assume that a has two rows. The third observation from 
above says that a satisfies the hypotheses of remark (6.2). Then (6.3) denies the 
existence of f3, contradiction. 

Assume now that d > 2 and that a has q + 1 = dim(S d ~ 1 V*) + 1 rows. Let us 
put a' = a mod Z, f3' = (3 mod Z and let us assume that the columns of f3' are 
linearly independent. If f3' has a zero row, then we may write 



(3 = 



hi 





/?22 



with a 1 x 3-matrix /?n that, according to observation two from above, is not zero. 
As all 3 x 3-minors of [3 are zero, we get det(/?22) = 0. Moreover, /3 2 2 cannot have 
linearly dependent rows or columns, otherwise observation one from above would 
be violated. Thus we may write 
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X 
Y 



u v 



with linearly independent u, v G V*. It follows that 



a 



an ai2 
a 2 \ 



with 



«12 



Vl 



J Q J 



[ -y x 



for some Vi, . . . , v q e S d ~ 1 V* . This contradicts observation three from above. Thus 
far we have reached the conclusion that (3' cannot have a zero row. As above, we 
may assume that 



P'l2 







%2 



with 



%2 



X 2 XY 
XY Y 2 
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and we get 







with a' 12 — 



v d 



[-Y X] 



for some homogeneous polynomials v\, . . . , Vd of degree d — 1 in X and Y. It follows 
that we can write 



a = 



ai 
Za 2 



with ct2 having q + 1 — d = 



d + 1 



+ l-d = 



1 rows. 



From the third observation at the beginning of this proof we see that a 2 satisfies 
the induction hypothesis. However, 012 violates the conclusion of the induction 
hypothesis, so we have arrived at a contradiction. 

It remains to examine the situation in which the columns of j3' are linearly 
dependent. Let us write 



P=[Z0! & ] With A 



u 

V 

w 



By the first observation at the beginning of this proof we see that at least two 
among u, v, w must be linearly independent, say u = X, v = Y and either w = Z 
or w = 0. From the relation 

" X ' 

a' Y =0 


we deduce, as before, the existence of a 2 , leading to a contradiction. This finishes 
the proof of the claim. 

Using similar considerations, which we omit, we can compute the remaining linear 
algebra constants: 



fc(l,ll) = 0, fc(2) = fc(l,7) 



d+1 
2 



fc(3) = fc(2,l) 



d + 2 



d+1 
2 



Conditions (7.3) now take the form 

A 2 > 6Ai, 

2n\2 > n\i + 



d+1 
d + 2 



d+1 



3nA 2 > n\i 
n\\ + n\ 2 > 
nXi + 2nX 2 > 

The second inequality follows from the first and the fourth. The third inequality 



d+1 

d + 2 



d+1 
2 
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follows from the first and the fifth. Using the relation Ai = 1 — 3A2 we see that the 
first inequality is equivalent to A2 > 6/19. From this and (7.2) we obtain that 

f 9 (d + 2 
n < — 



6 V 2 

It becomes now clear that the last three inequalities in (7.1) are superfluous, namely 
they are satisfied if we set A2 = 6/19 and n = 19(d 2 + 3d + 2)/12. Eliminating all 
unnecessary conditions from (7.1) and (7.3), we finally arrive at: 

(7.5) Claim: Let W be the space of morphisms of sheaves on P 2 of the form 

if : 0(-d - 2) © 30(-d) — ► nO. 
Then for any nonsingular polarization A satisfying 

6 1 

19 <A2< 3' 

A < 1 1 f d+1 

the set of semistable morphisms is nonempty and admits a geometric quotient 
W SS (G 1 A)/G, which is a quasiprojective variety. 

Note that necessarily n must satisfy the conditions 

19/d+l\ 19/d + 2\ 19/d+2 
13 1 2 ) + 2 ) <n< Y{ 2 

8. Morphisms of the Form mO(-di) © 0(-d 2 ) © 0(-d 3 ) — > nO 

We fix integers d\ > d 2 > d^ > 0, we fix a vector space V over k of dimension 
r + 1 and we consider morphisms of sheaves on P r = P(V) of the form 

ip = (<p\^ 2 ,p 3 ) : mO(-di) © 0{-d 2 ) © 0{-d 3 ) — »• nO. 

Employing the notations from [3] we have: 

mi = to, m 2 = 1, m 3 = 1, pi = m + a 2 i + a 31 , p 2 = 1 + 032, P3 = 1, 

toAi + A 2 + A 3 = 1, a.\ = Ai, a 2 = A 2 — a 2 iAi, a 3 = A3 - a 3 iAi - a 32 A 2 +a 32 a 2 iAi. 

The polarization A = (Ai, A 2 , A 3 , pi) is uniquely determined by the pair (Ai, A 2 ) in 
[0,1] x [0,1]. The singular polarizations are among those polarizations for which 
there are integers < k < n and < p < to such that 

- = p\i or -=p\ 1 + X 2 or - = p\ 1 + \ 3 . 

n n n 

Equivalently, for singular polarizations the pair (Ai,A 2 ) lies on one of the lines 
Ai = n/pn or A 2 = n/n — pX\. 



ON SOME QUOTIENTS MODULO NONREDUCTIVE GROUPS 



25 



Given integers < k\ < m, < k 2 < 1, < K3 < 1, we denote by l KlK2K3 the 
smallest integer satisfying 

^KiB 2 K 3 



> K1A1 + K 2 A 2 + K3A3 



and we consider matrices of the form 



0i K 



,.1- 



2,3. 



We put tp KlK2K3 = {<p\ lK2K3 ,<p 2 KlK2K3 ,<P% lK2K? )- According to King's Criterion of 
Semistability from [1] , the morphism ip is semistable if and only if it is not equivalent 
to a morphism of the form (p K1K2K3 for any choice of K\, k 2 , k 3 . 

We now turn to the embedding into the action of the reductive group. The map 
( : W — > W can be described explicitly: ((ip) — {£,2,£,3,"f(f)) where 

6 6M P1 , P2 (S J '-^*), 6eM P2 , P3 (^- ds r), 7 MeM„, Pl (S%'). 



Concretely, let us choose a basis {X , ■ ■ 

{Ui}l<i< a . 21 
{ V j}l<j<a 32 
{Wk}l<k<a 31 

■■Xt*. Then 



, X r } of V* and let us choose bases 
of S dl - d2 V*, 
of S d2 - d3 V*, 
of S dl ~ d3 V* 



made of monomials Xq° 




U 
w 



and 



where 



U=[U 1 



6 = [ Vx 



U a21 



Va 32 }' 



while W is an a^i xa3 2 -matrix with entries Wkj = Wk/Vj ifVj divides Wk, otherwise 
W k3 = 0. 

In order to apply the theory from section 2 we need to assume a priori that 
CK3 > and P1A1 < 1, see remark (2.7). The second condition follows from the 
condition a 2 > 0, which we will assume in the sequel. According to King's Criterion 
of Semistability, a point (£2, £3 , 7) G W is semistable if and only if the following 
conditions are satisfied: 

(i) (6,7) *° (fe)fcj,7ife) with l/n > k\ 1 +ia 2 + a 3 , < k < m + a 21 + a 3 i - 1, 
1 < i < 1 + 032- Here jik and (^ 2 )ki are matrices of the form 



Ilk 



0l,m+a 2 i+a 3 i— k * 



(£2) ki 



0fc,l+a32— i ^ 

(ii) (£,2, £3, 7) *° ((&)ki,{&)i,1lk), with l/n > fcAi + ia 2 , < k < m + a 2 i + 
031 — 1, l<i<l + 032. Here (£3)^ denotes a matrix having zero entries on 
the last i rows. 

(iii) (6,7) *° ((6)fe)7ife) with V n > < fe < m + a 2 i + a 3 i - 1. Here (£ 2 ) fe 
denotes a matrix with zero entries on the last k rows. 



(8.1) Remark: Let W be a matrix obtained by performing elementary row and 
column operations on W. Assume that W 1 has a zero submatrix with a 32 — 1 
columns. Then the zero submatrix has at most a 2 \ rows. 
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Proof: We notice first that the a 2 i nonzero entries of W on each column are linearly 
independent. The claim will follow if we show that the matrix Wj, obtained by 
deleting the j th column of W and those i th rows for which W^ ^ 0, has linearly 
independent rows. But, by construction, each row of Wj is not zero and the nonzero 
entries of Wj on each column are linearly independent. This finishes the argument. 

As a direct consequence of the above remark we get the following: 

(8.2) Remark: Assume that £2 ~ (^2)^1 ■ Then k < m + a 2 i. 

At the other extreme, we would like to know what is the largest k for which 
£2 ~ (^2)k,a 32 - F° r this we need the following analog of (8.1). Its proof will be 
included in the proof of (8.5): 

(8.3) Remark: Let W' be as at (8.1). Then each column of W' has a 2 \ linearly 
independent elements, in other words it spans S dl ~ d2 V*. 

(8.4) Remark: Assume that £2 ~ (&)k,a 32 - Then k < m + a 3 i. 

Given integers 1 < i < j < a 32 wc let be the number of nonzero rows of 
the matrix made of the columns i and j of W. Let to be the smallest among the 
numbers u^. Note that a 2 i < lo < 2a 2 i. In fact, we have the obvious formula 

^ = 2 ^di - d 2 + _ ^di - d 2 + r - 1^ 

(8.5) Remark: Let W' be as at (8.1). Assume that W' has a zero submatrix with 
two or more columns. Then the zero submatrix has at most 031 — to rows. 

Proof: We have to show that any matrix made of two columns of W' has w linearly 
independent rows. Let Wj be the j th column of W. Let Wj be a linear combination 
of the columns of W of the form 

l<3 

Given integers 1 < p < q < a 32 we have to show that the matrix W" made of the 
columns W' v and W' q has lo linearly independent rows. 

We choose the lexicographic ordering on the monomials Xq° ■ ■ ■ X 1 / that form 
a basis of S d2 ~ d3 V* . We also choose the lexicographic ordering on the monomials 
giving a basis for S dl ~ d3 V*. We write W relative to these orderings and we notice 
that, if the entry Wij of W is nonzero, then, for i < k, I < j, (i, j) ^ (k, I), Wki 
is either zero or is larger than Wij in the lexicographic ordering. This shows two 
things: 

(i) if Wij 7^ 0, then W[j is equal to Wij plus a linear combination of monomials 
that are larger than Wij in the lexicographic ordering; 

(ii) if Wij 7^ 0, then, for i < k < a 3 i, W' k j is either zero or is a combination of 
monomials larger than Wij . 
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Performing on Wj row operations of the form cRk + Ri — > Ri, i < k 1 c being a 
scalar, we do not disturb properties (i) and (ii). Moreover, performing a certain 
sequence of such operations, we can arrive at Wj. This proves remark (8.3). 

To show that W" has w linearly independent rows we proceed as follows. Per- 
forming, possibly, row operations on W" of the kind mentioned above, we may 
assume that W' v = W p . Now all we need to do is find u> — a 2 \ linearly independent 
elements among those W- for which W- = 0. But from (i) and (ii) we know that 
those W' iq for which Wi q ^ are linearly independent. As there are at least to — a 2 i 
indices i for which Wi q ^ but Wi P = 0, we are done. 

(8.6) Remark: Assume that £2 ~ {&)ki with i < a 32 — 1- Then k < m + a 2 \ + 

a-31 - u. 

Owing to the fact that £2 has 021 +031 linearly independent rows, we may assume 
that k < m in (iii). Owing to the fact that £3 has 032 linearly independent entries, 
we may assume that i = 1 in (ii) and, in view of (8.2), that k < m + a 2 i. According 
to (8.6), wc may assume that k < m + a 2 i + &31 — to in (i) if 2 < i < a 32 — 1. 
According to (8.4), we may assume that k < m + a^i in (i) if i = a% 2 . Thus, in 
order to show that (£2, £3, 7) is semistable, it is enough to show that 7 7^ 

with — > fcAi and < k < m, 
n 

or with — > fcAi + a 2 and m < k < m + a 2 i, 
n 

or with — > fcAi + 20:2 + «3 and m + a 2 i < k < m + a 2 i + a^i — u>, 



n 



2 < i < a 32 - 1 



or with — > fcAi + a 32 o:2 + a 3 and m + a 2 i + 0131 — ui < k < m + 031, 

n 

or with — > fcAi + (a 32 + l)a 2 + a 3 and m + a 3 i < k < m + a 2 \ + CL31 — 1. 
n 

(8.7) Claim: Let W be the space of morphisms of sheaves on P r of the form 

LP : mO(-di) © 0{-d 2 ) © 0(-d 3 ) — )• nO. 
Assume that m < to — a 21 , in other words assume that 

m< { r-l 

Then for any nonsingular polarization A = (Ai, A2, A3, /xi) satisfying 

1 - mAi - d3iAi + a 3 2a 2 iAi 

a 2 iAi < A 2 < — , 

1 + 032 

A2 < 1 — mAi, 

, . 1 1 fd 3 +r 



m + 021 mn + 021?^ \ r 

the set of semistable morphisms admits a geometric quotient W ss (G, A)/G, which 
is a quasiprojective variety. 
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Proof: Let ip be in W SS {G 1 A). According to (2.6), we need to show that (£2, £3, 7(/)) 
is semistable. We argue by contradiction. Assume that j(ip) ~ 7^ with i/n > fcAi 
and < fc < m. Let ip — (ip 1 , ip 2 , ip 3 ) denote the truncated matrix consisting of the 
first I rows of ip. By assumption ip has kernel inside k m S^i-^y* gd 1 -d 3 y* of 
dimension at least m + 0,21 + &3i — k, which is greater than m + a 2 \ and m + a 3 i 
because, by hypothesis, m < 021 and m < a^\. This shows that Ker(^) intersects 
S d i- d2 V* and S dl - d3 V* nontrivially, forcing ip 2 = and ip 3 = 0. Moreover, re- 
placing possibly ip with an equivalent morphism, we may assume that there are at 
least m — k linearly independent elements in the kernel of if) 1 , viewed as a subspace 
of k m . We get tp <~ (pk,a.o, contradicting the semistability of ip. 

Assume now that -f(<p) ~ -fit with l/n > fcAi + a 2 and m < k < m + a 2 \. Note 
that automatically I > l m .o,o- This excludes those -fit with k < 021 because, as we 
saw above, the condition fc < a 2 i forces i/^ 1 = 0, ip 2 — 0, so it yields ip ~ </?m,o,o> 
which is a contradiction. Thus CI21 < fc < a 2 i + m. We have 

ker(i/> 3 ) > — fc > a3i — a2i — m > a^i — u> > 0, 

hence V' 3 = 0. We cannot have ip 2 — because this would lead to the contradiction 
¥ ~ i^ m ,o ; o- Thus the elements from Kev(ip) project onto m + a 2 i — fc linearly 
independent elements in k m . As l/n > (fc — a 2 i)Ai +A 2 , we obtain the contradiction 

</> ~ fk-a 2u l,0- 

Assume that -f(ip) ^ "fik with^/n> k\i+ia 2 +a 3 , m+a 2 i < fc < m+a 2 i+a 3 i— cj 
and 2 < z < &32 — 1- Notice that automatically I > Z mi i.o, so we cannot have i/; 3 = 0, 
because this would lead to the contradiction tp ~ </5 m ,i,o- Thus Ker(^) intersects 
^di-day* trivially, forcing 

m + 021 + CL31 — k < m + a 2 i, so 031 < fc, so a 3 i < m + a 2 i + a 31 — lo. 

This contradicts the hypothesis on to. 

We next examine the case "f(ip) ~ 7;fc with ^/n > fcAi + 0320:2 + 03 = {k — 
«3i)Ai + A3 and to + a 2 \ + &31 — uj < k < m + C131. As before, ip 3 ^ 0. Thus 

m + a 2 i> ker(?/' 1 , 1b 2 ) > to + a 2 i + a 3 i — fc > a 2 i > cj — a 2 i > to. 

This shows that fc > 031 and that Ker(^) intersects g d i- d 2y* nontrivially. Replac- 
ing possibly ip with an equivalent morphism, we may assume that if) 2 = and that 
ker('0 1 ) > m + a 2 i + a 3 i — fc — a 2 i = to + a 3 i — fc. We arrive at the contradiction 

f ~ V/c-031,0,1- 

The last situation we need to examine is -f(<p) ~ 7;^ with 

- > fcAi + (a 32 + l)a 2 + a 3 = (fc - a 2 i - a 3 i)Ai + A 2 + A 3 
n 

and m + a 3 i < fc < m + a 2 \ + 0,31 — 1. Notice that automatically / > l m fi,i, so 
Ker(^) intersects S dl ~ d2 V* and g d i- d 3y* trivially, otherwise we would get the 
contradictions ip ~ ^m,i,o or ip ~ i/? m ,oa- 

Assume that (/, g) is a nonzero vector in Ker(ip 2 , ip 3 ) viewed as a subspace of 
gdi-d 2 y* ^ gdx-dzy* _ ^ e saw a k ove that both / and 5 must be nonzero, so we 

can write / = hf\ , g = hg\ with /1 , 51 relatively prime and 

max{0, di — d 2 — g^} < d — dcg(h) < d\ — d 2 . 

The rows of {ip 2 , ip 3 ) are of the form (— g\u, fiu), hence they are vectors in a vector 
space of dimension equal to dim(S d2+d:i+d ~ dl V*). Had we had the inequality I > 
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l m ,o,o + <lim(S d2+d3+d dl V*), we would arrive at the contradiction ip <~ (p m ,o,o- But 
this inequality follows from the condition 

n(a 3 iAi + (a 32 + l)o 2 + a 3 ) > dim^V*) 

which is equivalent to the last condition from the statement of the claim. 

In conclusion, Ker(^) intersects trivially S dl ~ d2 V* © S dl ~ ds V*, hence its ele- 
ments project onto linearly independent elements in k m . We get m > ker(^) > 
fn + &21 + &31 — k forcing k > a 2 i + 031- We arrive at the contradiction ip ~ 
Vfe-02i-o 3 i,i,i- This finishes the proof of the claim. The remaining conditions from 
the statement of the claim are there to ensure that A3 > and 0:3 > 0. 

According to corollary 7.2.2 in [3], if a 2 > 0, a 3 > and if A 2 > a 2 ici(l, l)/n, 
then the conclusion of (8.7) holds. The constant Ci(l, 1) can be computed as in the 
proof of lemma 9.1.2 (loc. cit.) One has 

dim(S d *V) 
dim(S d ^V)' 

Thus, according to Drezet and Trautmann, the conclusion of (8.7) holds under the 
hypotheses 

1 - m\ 1 - a 3 iAi + a 32 a 21 \ 1 
a 2 iAi < A 2 < — , 

1 + «32 

A 2 < 1 - mAi, 
na 3 i V r 

Our result is not contained in Drezet and Trautmann's result. 
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